1. Field of the Invention
The present invention generally relates to sound image localization method/apparatus and also a sound image control apparatus. More specifically, the present invention is directed to a sound image localization apparatus and a sound image localization method, capable of localizing a sound image at an arbitrary position within a three-dimensional space, which are used in, for instance, electronic musical instruments, game machines, and acoustic appliances (e.g. mixers). Also, the present invention is directed to a delay amount control apparatus for simulating an inter aural time difference changed in connection with movement of a sound image based upon variation of a delay amount, and also to a sound image control apparatus for moving a sound image by employing this delay amount control apparatus.
2. Description of the Related Art
Conventionally, such a technical idea is known in the field that 2-channel stereophonic signals are produced, and these stereophonic signals are supplied to right/left speakers so as to simultaneously produce stereophonic sounds, so that sound images may be localized. In accordance with this sound image localization technique, the sound images are localized by changing the balance in the right/left sound volume, so that the sound images could be localized only between the right/left speakers.
To the contrary, very recently, several techniques have been developed by which sound images can be localized at an arbitrary position within a three-dimensional space. As one of sound image localization apparatus using this conventional sound image localization technique, an input signal is processed by employing a head related acoustic transfer function so as to localize a sound image. In this case, a head related acoustic transfer function implies such a function for indicating a transfer system defined by such that a sound wave produced from a sound source receives effects such as reflection, diffraction, and resonance caused by a head portion, an external ear, a shoulder, and so on, and then reaches an ear (tympanic membrane) of a human body.
In this conventional sound image localization apparatus, when sounds are heard by using a headphone, first to fourth head related acoustic transfer functions are previously measures. That is, the first head related acoustic transfer function of a path defined from the sound source to a left ear of an audience is previously measured. The second head related acoustic transfer function of a path defined from the sound source to a right ear of the audience is previously measured. The third head related acoustic transfer function of a path defined from a left headphone speaker to the left ear of the audience is previously measured, and the fourth head related acoustic transfer function of a path defined from the right headphone speaker to the right ear of this audience is previously measured. Then, the signals supplied to the left headphone speaker are controlled in such a manner that the sounds processed by employing the first head related acoustic transfer function and the third head related acoustic transfer function are made equal to each other near the left external ear of the audience. Also, the signals supplied to the right headphone speaker are controlled in such a manner that the sounds processed by employing the second head related acoustic transfer function and the fourth head related acoustic transfer function are made equal to each other near the right external ear of the audience. As a consequence, the sound image can be localized at the sound source position.
When the sounds are heard by using speakers, head related acoustic transfer functions of paths defined from the left speaker to the right ear and from the right speaker to the left ear are furthermore measured. While employing these head related acoustic transfer functions, the sounds which pass through these paths and then reach the audience (will be referred to as xe2x80x9ccrosstalk soundsxe2x80x9d hereinafter) are removed from the sounds produced by using the speakers. As a consequence, since a similar sound condition to that of the headphone can be established, the sound image can be localized at the sound source position.
One example of the above-described conventional sound image localization apparatus is shown in FIG. 1. In FIG. 1, a data memory 50 stores a plurality of coefficient sets. Each coefficient set is constructed of a delay coefficient, a filter coefficient, and an amplification coefficient. Each of these coefficient sets corresponds to a direction of a sound source as viewed from an audience, namely a direction (angle) along which a sound image is localized. For instance, in such a sound image localization apparatus for controlling the sound image localization direction every 10 degrees, 36 coefficient sets are stored in this data memory. The externally supplied sound image localization direction data may determine which coefficient set is read out from this data memory. Then, the delay coefficient contained in the read coefficient set is supplied to a time difference signal producing device 51, the filter coefficient is supplied to a left head related acoustic transfer function processor 52 and also to a right head related acoustic transfer function processor 53, and further the amplification coefficient is supplied to a left amplifier 54 and a right amplifier 55.
The time difference signal producing device 51 is arranged by, for example, a delay device, and may simulate a difference between a time when a sound produced from a sound source reaches a left ear of an audience, and another time when this sound reaches a right ear of this audience (will be referred to as an xe2x80x9cinter aural time differencexe2x80x9d hereinafter). For example, both a monaural input signal and a delay coefficient are inputted into this time difference signal producing device 51.
In this case, a direction of a sound source as viewed from an audience, namely a direction (angle) along which a sound image is localized will now be defined, as illustrated in FIG. 2. In this case, it is assumed that a front surface of the audience is a zero (0) degree. In general, an inter aural time difference becomes minimum when the sound source is directed to the zero-degree direction, is increased while the sound source is changed from this zero-degree direction to a 90-degree direction, and then becomes maximum in the 90-degree direction. Furthermore, the inter aural time difference is decreased while the sound source is changed from this 90-degree direction to a 180-degree direction, and then becomes minimum in a 180-degree direction. Similarly, the inter aural time difference is increased while the sound source is changed from the 180-degree direction to a 270-degree direction, and then becomes maximum in this 270-degree direction. The inter aural time difference is decreased while the sound source is changed from the 270-degree direction to the zero-degree (360-degree) direction, and then becomes minimum in the zero-degree direction again. The delay coefficients supplied to the time difference signal producing device 51 own values corresponding to the respective angles.
When the sound image localization direction data indicative of a degree larger than, or equal to 0 degree, and smaller than 180 degrees is inputted, the time difference signal producing device 51 directly outputs this input signal (otherwise delays this input signal only by a predetermined time) as a first time difference signal, and also outputs a second time difference signal delayed from this first time difference signal only by such an inter aural time difference corresponding to the delay coefficient. Similarly, when the sound image localization direction data indicative of a degree larger than, or equal to 180 degrees, and smaller than 360 degrees is inputted, the time difference signal producing device 51 directly outputs this input signal (otherwise delays this input signal only by a predetermined time) as a second time difference signal, and also outputs a first time difference signal delayed from this second time difference signal only by such an inter aural time difference corresponding to the delay coefficient. The first time difference signal produced from the time difference signal producing device 51 is supplied to the left head related acoustic transfer function processor 52, and the second time difference signal produced therefrom is supplied to the right head related acoustic transfer function processor 53.
The left head related acoustic transfer function processor 52 is arranged by, for instance, a six-order FIR filter, and simulates a head related acoustic transfer function of a sound entered into the left ear of the audience. The above-described first time difference signal and a filter coefficient for a left channel are entered into this left head transfer function processor 52. The left head related acoustic transfer function processor 52 convolutes the impulse series of the head related acoustic transfer function with the input signal by employing the filter coefficient for the left channel as the coefficient of the FIR filter. The signal processed from this left head related acoustic transfer function processor 52 is supplied to an amplifier 54 for the left channel.
The right head related acoustic transfer function processor 53 simulates a head related acoustic transfer function of a sound entered into the right ear of the audience. The above-described second time difference signal and a filter coefficient for a right channel are entered into this right head transfer function processor 53, which is different from the left head related acoustic transfer function processor 52. Other arrangements and operation of this right head related acoustic transfer function processor 53 are similar to those of the above-explained left head related acoustic transfer function processor 52. A signal processed from this right head related acoustic transfer function processor 53 is supplied to an amplifier 55 for a right channel.
The amplifier 54 for the left channel simulates a sound pressure level of a sound entered into the left ear of the audience, and outputs the simulated sound pressure level as the left channel signal. Similarly, the amplifier 55 for the right channel simulates a sound pressure level of a sound entered into the right ear, and outputs the simulated sound pressure level as the right channel signal. With employment of this arrangement, for instance, when the sound source is directed along the 90-degree direction, the sound pressure level of the sound entered into the left ear becomes maximum, whereas the sound pressure level of the sound entered into the right ear becomes minimum.
In accordance with the sound image localization apparatus with employment of above-explained arrangement, when the sounds are heard by using the headphone, no extra device is additionally required, whereas when the sounds are heard by using the speakers, the means for canceling the crosstalk sounds is further provided, so that the sound image can be localized at an arbitrary position within the three-dimensional space.
However, since the left head related acoustic transfer function processor and the right head related acoustic transfer function processor are separately provided in this conventional sound image localization apparatus, 12-order filters are required in total. As a result, in such a case that these right/left head related acoustic transfer function processors are constituted by using the hardware, huge amounts of delay elements and amplifiers are required, resulting in the high-cost and bulky sound image localization apparatus. In the case that the right/left head related acoustic transfer function processors are constituted by executing software programs by a digital signal processor (will be referred to as a xe2x80x9cDSPxe2x80x9d hereinafter), a very large amount of processing operations is necessarily required. As a consequence, since such a DSP operable in high speeds is required so as to process the data in real time, the sound image localization apparatus becomes high cost.
Furthermore, since the coefficient sets must be stored every sound image localization direction, such a memory having a large memory capacity is required. To further control the direction (angle) along with the sound image is localized in order to improve the precision of the sound image localization, a memory having a further large memory capacity is needed. There is another problem that the real time data processing operation is deteriorated, because the coefficient sets must be replaced every time the direction along which the sound image is localized is changed.
On the other hand, another conventional sound image localization apparatus capable of not only localizing the sound image, but also capable of moving the sound image has been developed. As such an apparatus to which the technique for moving the sound image has been applied, for instance, Japanese Laid-open Patent Application (JP-A-Heisei 04-30700) discloses the sound image localization apparatus. This disclosed sound image localization apparatus is equipped with sound image localizing means constituted by delay devices and higher-order filters. The head related acoustic transfer function is simulated by externally setting the parameters arranged by the delay coefficient and the filter coefficient. This head related transfer coefficient will differ from each other, depending upon the localization positions of the sound image as viewed from the audience. Therefore, in order that the sound image is localized at a large number of positions, this conventional sound image localization apparatus owns a large quantity of parameters corresponding to the respective localization positions.
In general, when a localization position of a sound image is moved from a present position to a new position, a parameter corresponding to this new position may be set to the sound image localization means. However, if the parameter is simply set to the sound image localization means while producing the signal, then discontinuous points are produced in the signal under production, which causes noise. To avoid this problem, this conventional sound image localization apparatus is equipped with first sound image localization means and second sound image localization means, and further means for weighting the output signals from the respective sound image localization means by way of the cross-fade system.
It is now assumed that the sound image is localized at the first position in response to the first localization signal derived from the first sound image localization means. When this sound image is moved to the second position, the weight of xe2x80x9c1xe2x80x9d is applied to the first localization signal derived from the first sound image localization means, and also the weight of xe2x80x9c0xe2x80x9d is applied to the sound localization signal derived from the second sound image localization means. Under these conditions, the parameter used to localize the sound image to the second position is set to the second sound image localization means. Since the second localization signal is weighted by xe2x80x9c0xe2x80x9d, there is no possibility that noise is produced in the second localization signal when the parameter is set.
The weight of the first localization signal is gradually decreased from this state, and further the weight of the second localization signal is gradually increased. Then, after a predetermined time has elapsed, the weight to be applied to the first localization signal is set to xe2x80x9c0xe2x80x9d, and the weight to be applied to the second localization signal is set to xe2x80x9c1xe2x80x9d. As a result, moving of the sound image from the first position to the second position is completed without producing the noise.
The above-described sound image moving process is normally carried out by employing, for example, a DSP. In this case, the digital input signal is entered into the first and second sound image localization means every sampling time period. As a result, this DSP must process a single digital signal within a single sampling time period. For example, if the input signal is obtained by being sampled at the frequency of 48 kHz, the sampling time period becomes approximately 21 microseconds. Therefore, this DSP must perform the following process operation every approximately 21 microseconds, namely, the first localization signal is produced and weighted, and the second localization signal is produced and weighted. After all, there is another problem that the high cost DSP operable in high speeds is necessarily required in this conventional sound image localization apparatus.
As a consequence, an object of the present invention is to provide a sound image localization apparatus and a sound image localizing method, capable of localizing a sound image at an arbitrary position within a three-dimensional space with keeping a superior real-time characteristic by employing a simple/low-cost circuit, or a simple data processing operation.
Another object of the present invention is to provide a delay amount control apparatus capable of changing a delay amount in high speed without producing noise.
A further object of the present invention is to provide a sound image control apparatus capable of changing a delay amount without producing noise, and therefore capable of moving a sound image in high speed and in a smoothing manner.
To achieve the above-described objects, a sound image localization apparatus for producing a first channel signal and a second channel signal, used to localize a sound image, according to a first aspect of the present invention, comprising:
time difference signal producing means for sequentially outputting externally supplied input signals as a first time difference signal and a second time difference signal while giving an inter aural time difference corresponding to a sound image localization direction, wherein the second time difference signal is outputted as a second channel signal; and
function processing means for processing the first time difference signal derived from the time difference signal producing means with employment of a relative function constituted by a ratio of a left head related acoustic transfer function to a right head related acoustic transfer function in response to the sound image localization direction, and outputting a processed signal as a first channel signal.
The respective means for constituting the sound image localization apparatus according to the first aspect of the present invention, a delay amount control apparatus according to a third aspect of the present invention (will be explained later), and a sound image control apparatus according to a fourth aspect of the present invention (will be described later) may be realized by employing a hardware, or by executing a software processing operation by a DSP, a central processing unit (CPU), and the like.
The externally supplied input signal contains, for instance, a voice signal, a music sound signal, and so on. This input signal may be formed as, for example, digital data obtained by sampling an analog signal at a preselected frequency, by quantizing the sampled signal, and further by coding this quantized sampled signal (will be referred to as xe2x80x9csampling dataxe2x80x9d hereinafter). This input signal is supplied from, for example, an A/D converter every sampling time period.
The time difference signal producing means may be arranged by, for instance, a delay device. To this time difference signal producing means, for example, a monaural signal may be entered as the input signal. In such a case that the first time difference signal outputted from this time difference signal producing means is used as the left channel signal, if the sound image localization direction is larger than, or equal to 0 degree and smaller than 180 degrees, then the first time difference signal is first outputted, and subsequently the second time difference signal is outputted which is delayed only by the inter aural time difference with respect to this first time difference signal. This inter aural time difference is different from each other, depending on the direction of the sound source as viewed from the audience, namely the sound image localization direction (angle).
If the sound image localization direction is larger than, or equal to 180 degrees and smaller than 360 degrees, then the second time difference signal is first outputted, and subsequently the first time difference signal is outputted which is delayed only by the inter aural time difference with respect to this second time difference signal. When the first time difference signal is used as the right channel signal, the output sequence of the first time difference signal and the second time difference signal is reversed as to the above-described output sequence.
The relative function used in the function processing means is constituted by a ratio of the left head related acoustic transfer function to the right head transfer related transfer function in the conventional sound image localization apparatus. Conceptionally speaking, this relative function may be conceived as such a function obtained by dividing each of the functions used in the left head related acoustic transfer function processor 52 and the right head related acoustic transfer function processor 53 shown in FIG. 1 by the function used in the right head related acoustic transfer function processor 53. As a result, only the first time difference signal is processed in the function processing means, and the second time difference signal is directly outputted as the second channel signal.
Since the function processing means is arranged in the above-described manner, the process operation for applying the head related acoustic transfer function only to the first time difference signal is merely carried out, and there is no need to carry out the process operation for the second time difference signal. As a consequence, when this sound image localization apparatus is arranged by, for example, hardware, a total amount of hardware can be reduced. When this sound image localization apparatus is arranged by executing software processing operation, a total calculation amount can be reduced.
Also, the image localization apparatus according to the first aspect of the present invention may be arranged by further comprising:
correcting means constructed of a filter for filtering the externally supplied input signal, a first amplifier for amplifying a signal filtered out from the filter, a second amplifier for amplifying the externally supplied input signal, and an adder for adding an output signal from the first amplifier to an output signal from the second amplifier, wherein the correcting means controls gains of the first amplifier and of the second amplifier to thereby correct sound qualities and sound volumes of sounds produced based upon the first channel signal and the second channel signal. This correcting means may be provided at a prestage, or a poststage of the time difference signal producing means. Preferably, this correcting means is provided at the prestage of the time difference signal producing means.
In the sound image localization apparatus according to the first aspect of the present invention, the relative function made of the ratio of the left head related acoustic transfer function to the right head related acoustic transfer function is utilized as the head related acoustic transfer function used to localize the sound image. As a result, in such a case that the sound image is localized near the 90-degree direction and the 270-degree direction where the ratio of the right/left head related acoustic transfer functions is large, the sound quality is greatly changed. On the other hand, in such a case that the sound image is localized near the 0-degree direction and the 180-degree direction where the ratio of the right/left head related acoustic transfer functions is small, no clear discrimination can be made as to whether the sound image is localized in the front direction (namely, 0-degree direction), or in the rear direction (namely, 180-degree direction). Therefore, unnatural feelings still remain. To solve such a problem, the correcting means corrects the input signal so as to achieve such a frequency characteristic close to the original frequency characteristic, so that a change in the sound quality can be suppressed. Also, since the sound volume is excessively increased near the 90-degree direction and the 270-degree direction, the correcting means corrects the sound volume in order to obtain uniform sound volume feelings. Since such a sound volume correction is carried out, unnatural feelings in the sound qualities and sound volume can be removed.
The respective gains of the first amplifier and the second amplifier contained in this correcting means may be controlled based upon data calculated in accordance with a preselected calculation formula. In this case, as this preselected calculation formula, a linear function prepared for each of these first and second amplifiers may be employed. According to this arrangement, the data used to control the respective gains of the first amplifier and the second amplifier need not be stored every sound image localization direction, so that a storage capacity of a memory can be reduced. This memory should be provided in an apparatus to which this sound image control apparatus is applied.
Also, the image localization apparatus according to the first aspect of the present invention may be arranged by further comprising:
time difference data producing means for producing inter aural time difference data in accordance with a preselected calculation formula, the inter aural time difference data is used to produce an inter aural time difference in response to the sound image localization direction, wherein the time difference signal producing means sequentially outputs the first time difference signal and the second time difference signal, while giving an inter aural time difference corresponding to the inter aural time difference data produced by the time difference data producing means.
Above-described function processing means may include:
a plurality of fixed filters into which the first time difference signal is inputted;
a plurality of amplifiers for amplifying signals filtered out from the respective fixed filters; and
an adder for adding signals derived from the plurality of amplifiers to each other, wherein
the function processing means controls each of gains of the plural amplifiers to simulate the relative function.
In this case, second order IIR type filters may be used as the plurality of fixed filters.
Also, to achieve the above-described objects, a sound image localizing method, according to a second aspect of the present invention, comprising the steps of:
sequentially outputting externally supplied input signals as a first time difference signal and a second time difference signal while giving an inter aural time difference corresponding to a sound image localization direction;
processing the first time difference signal by employing a relative function made of a ratio of a left head related acoustic transfer function to a right head related acoustic transfer function in response to the sound image localization direction, whereby a first channel signal is produced; and
localizing a sound image based upon the first channel signal and the second time difference signal functioning as a second channel signal.
This sound image localizing method may be arranged by further comprising the step of:
adding a signal obtained by filtering the externally supplied input signal and amplifying the filtered input signal to another signal obtained by amplifying the externally supplied input signal, wherein sound qualities and sound volumes of sounds produced based on the first channel signal and the second channel signal are corrected by controlling gains of both the amplification for the filtered input signal and the amplification for the externally supplied input signal. In this case, the gains of the amplification for the filtered input signal and of the amplification for the externally supplied input signal may be determined in accordance with a predetermined calculation formula.
Also, the sound image localizing method may be arranged by further comprising the step of:
producing inter aural time difference data used to produce an inter aural time difference corresponding to the sound image localization direction in accordance with a preselected calculation formula, wherein in the outputting step , the first time difference signal and the second time difference signal are sequentially outputted while giving an inter aural time difference corresponding to the inter aural time difference data produced at the time difference data producing step.
Above-described step for producing the first channel signal may include:
filtering the first time difference signal by using a plurality of fixed filters, amplifying each of the filtered first time difference signals, and adding the amplified first time difference signals, whereby the relative function may be simulated by controlling the gains of the amplification for the filtered input signal and of the amplification for the externally supplied input signal.
Also, to achieve the above-described objects, a delay amount control apparatus for delaying an externally supplied input signal based on an externally supplied delay coefficient to output a delayed input signal, according to a third aspect of the present invention, comprising:
delay amount detecting means for detecting as to whether or not the delay coefficient is changed;
delay amount saving means for saving a delay coefficient before being changed when the delay amount detecting means detects that the delay coefficient is changed;
delay means for outputting a first delay signal produced by delaying the externally supplied input signal by delay amount designated by the delay coefficient before being changed, which is saved in the delay amount saving means, and also a second delay signal produced by delaying the externally supplied input signal by a delay amount designated by the externally supplied delay coefficient; and
cross-fade mixing means for cross-fading the first delay signal and the second delay signal outputted from the delay means so as to mix the first delay signal with the second delay signal.
The delay means may be constructed of, for instance, a memory. This memory sequentially stores sampling data corresponding to the externally entered input signals. In this case, the delay coefficient used to designate the delay amount may be constituted by an address used to read the sampling data from this memory. The delay amount is determined based on this address value. It should also be noted that the delay means may be constituted by a delay line element provided outside the DSP. In this case, the delay coefficient is used to select the output tap of this delay line element.
The delay amount saving means saves, for instance, an address as a delay coefficient before being changed. The cross-fade mixing means cross-fade-mixes the sampling data sequentially read out from the memory in response to the addresses saved in this delay amount saving means, and the sampling data sequentially read out from the memory in response to the newly applied address. In other words, the first delay signal delayed only by the delay amount designated by the delay coefficient before being changed is cross-fade-mixed with the second delay signal delayed only by the delay amount designated by the delay coefficient after being changed.
The above-described cross-fade mixing means may sequentially add the first delay signal decreased within a preselected time range to the second delay signal increased within the preselected time range. Concretely speaking, the first delay signal is multiplied by a coefficient xe2x80x9cBxe2x80x9d which is decreased while time has passed, and the second delay signal is multiplied by another coefficient (1-B) which is increased while time has passed. Then, the respective multiplied results are added to each other. In this case, the respective coefficient values are selected in such a manner that the addition result obtained by adding the coefficient B to the coefficient 1-B continuously becomes a constant value (for instance xe2x80x9c1xe2x80x9d). Even when the delay coefficient is changed, since the input signal is outputted which has been delayed only by the gently changed delay amount by way of this cross-fade mixing operation, no discontinuous point is produced in the signal. As a consequence, no noise is produced.
Also, to achieve the above-described objects, a sound image control apparatus for producing sounds in response to a first channel signal and a second channel signal so as to localize a sound image, according to the fourth aspect of the present invention, comprising:
delay amount control means for delaying an externally supplied input signal based upon a delay coefficient indicative of an inter aural time difference corresponding to a second image localization direction to thereby output the delayed externally supplied input signal;
first function processing means for processing the input signal in accordance with a first head related acoustic transfer function to thereby output the processed input signal as the first channel signal; and
second function processing means for processing the delayed input signal derived from the delay amount control means in accordance with a second head related acoustic transfer function to thereby output the processed delayed input signal as the second channel signal, wherein
the delay amount control means is composed of:
delay amount detecting means for detecting as to whether or not the delay coefficient is changed;
delay amount saving means for saving a delay coefficient before being changed when the delay amount detecting means detects that the delay coefficient is changed;
delay means for outputting a first delay signal produced by delaying the externally supplied input signal by a delay amount designated by the delay coefficient before being changed, which is saved in the delay amount saving means, and also a second delay signal produced by delaying the externally supplied input signal by a delay amount designated by the externally supplied delay coefficient; and
cross-fade mixing means for cross-fading the first delay signal and the second delay signal outputted from the delay means so as to mix the first delay signal with the second delay signal.
This sound image control apparatus may be arranged by further comprising:
storage means for storing therein both a delay coefficient and an amplification coefficient in correspondence with a sound image localization direction, wherein when the sound image localization direction is externally designated, the delay coefficient read from the storage means is supplied to the delay amount detecting means and the delay means included in the delay amount control means.
In this sound image control apparatus, each of the first function processing means and the second function processing means may include:
a plurality of fixed filters for filtering inputted signals with respect to each of frequency bands;
a plurality of amplifiers for amplifying signals filtered out from the respective fixed filters; and
an adder for adding signals amplified by the plurality of amplifiers, wherein each of gains of the plural amplifiers is controlled so as to simulate the first and second head related acoustic transfer functions. In this case, second order IIR type filters may be used as the plurality of fixed filters.
Also, the sound image control apparatus may be arranged by further comprising:
storage means for storing therein both a delay coefficient and an amplification coefficient in correspondence with a sound image localization direction, wherein when the sound image localization direction is externally designated, the amplification coefficient read from the storage means is supplied to the amplifiers included in the first function processing means and the second function processing means.